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ErrECT or Computer-Assisted Instruction (CAD on Reading 
Achievement: A Meta-Analnsis 



By Kyaw St*. PhD. Sian Koki. andJuvenna SI. Chang. Ed.D 



Abstract 



Whether computer-assisted instruction (CAI) can improve 
reading achievement of students has been a crucial ques- 
tion addressed by studies in the past. This meta-analys.s 
reviewed 17 research studies based on students K.-12 an 
revealed that CAI does have a positive effect on reading 
achievement. Although the effects of CAI in 17 studies 
were not homogeneous, there seems to be no particular 
study characteristic that might have caused the hetero- 
geneity. 
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Introduction 



Background 

'T’here is general agreement that reading is essential to success in our society. The ability to read is high- 
ly valued and important for social and economic advancement (Snow. Bums, & Griffin, 1998). The 
consensus supports the belief that reading is fundamental. 

Most children learn to read fairly well. However, there are children in America whose educational con- 
cerns are at risk because they do not read well enough to ensure understanding or to meet the demands of 
an increasingly competitive economy and changing demographics. It is the opinion of Pacific educators 
that not all of the children in Pacific schools are learning to read as well as they should. Many of them are 
experiencing serious difficulty in learning to read, and as they progress through the grades, they continue 
to lag in reading achievement. 

While the actual number of children who are poor readers is being debated, one widely accepted indicator 
is that 40 percent of all U.S. nine-year-olds score below the “basic" level on the National Assessment of 
Educational Progress (National Center for Education Statistics [NCES], 1999b). However "poor reader” is 
defined, the number of poor readers in our midst is too high (McPike, 1998). 

According to Learning First Alliance, the reading problems of U.S. children are not new. Overall reading 
performance has remained about the same since the first NAEP report was issued. Clearly, our children 
still need much support in learning to read and in using reading as a tool for learning (Snow. Bums, & 
Griffin, 1998). 

Computer-Assisted Instruction (CAI) is among the range of strategies being used to improve student 
achievement in school subjects, including reading. Programs for CAI have come a very long way since 
they were first developed over two decades ago. These programs tutor and drill students, diagnose prob- 
lems, keep records of student progress, and present material in print and other manifestations. It is 
believed that they reflect what good teachers do in the classroom (Kulik, Bangert, & Williams. 1983). 

Students are expected to benefit from CAI. Among the benefits that have been expected are better and 
more comfortable learning for students, since they learn at their own pace and convenience; opportunities 
to work with vastly superior materials and more sophisticated problems; personalized tutoring; automatic 
measurement of progress; and others. 

Teachers as well are expected to gain from CAI, as they experience less drudgery and repetition, greater 
ease in updating instructional materials, more accurate appraisal and documentation of student progress, 
and more time to work directly with students (Kulik, Bangert. & Williams. 1983). With increasing 
advances in computer technology, computer-assisted instruction (CAI) is now seen by many as a method 
of providing relevant instruction for large numbers of students. 

A number of different approaches to the use of computers in education are reflected in educational prac- 
tices. A useful classification of these approaches is that of Goldberg and Sherwood (1983). Of these cate- 
gories— Learning from computers. Learning about computers, and Learning about thinking with comput- 
ers — the most relevant to this study is Learning from computers. 

Learning from computers encompasses approaches to CAI in which the computer is used as a means for 
transmitting specific subject matter, such as reading. The flow of information is basically from the com- 
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puter to the student, with the computer presenting learning material or activities for student responses. The 
computer retains records of the student's progress through the course of study. Based on the degree of 
interaction between student and computer, researchers have identified three levels of CAI: 

Drill and practice : The computer provides the student with exercises that reinforce the learning of specif* 
ic skills taught in the classroom, and supplies immediate feedback on the correctness of the response. 

Used in this manner, CAI functions as a supplement to regular classroom instruction, and may be espe- 
cially useful when a teacher does not have the time to work individually with each student. Drill and prac- 
tice on the computer may also motivate students more than traditional workbook exercises. 

Tutorial : Tutorial CAI provides some information or clarifies certain concepts in addition to providing the 
student with practice exercises. In this sense, the computer begins to take over actual instructional func- 
tions, tailored to the student's individual level of achievement. 

Dialogue : With this type of computer use. the student takes an active role in interacting with the comput- 
er, giving instructions in the form of a computer language so as to structure the student’s own curriculum. 
The computer provides information, exercises, and feedback. Dialogue CAI is believed to come closest to 
actually substituting for regular instruction (Gourgey, Azumi, Madhere, & Walker. 1984). 

The verdict for the use of computers in education seems to be in. As stated by the National Center for 
Education Statistics (NCES): 

Computers have become an essential tool in our society . Early exposure to computers may help 
students gain the computer literacy that will be crucial for future success in the workplace. 

Access to computers at school and at home allows students to retrieve information, manipulate 
data . and produce results efficiently and in innovative ways. Examining the extent to which stu- 
dents have access to computers at school and at home may be an indicator of how well-prepared 
students will be to enter an increasingly technological workplace. (NCES. 1999a , p.64) 

Has computer-assisted instruction (CAI) produced benefits that result in greater achievement for students , in this 
case in reading? 



Soon after the introduction of CAI, educational researchers began to develop evaluation studies to answer 
this question. Although these evaluation studies produced potentially useful information on the effects of 
CAI, their messages were shrouded in ambiguity. One reason for unclear messages was that each evalua- 
tion report was published separately, making the total picture somewhat murky. 

Another problem had a deeper and more serious nature. These evaluation studies were never exact repli- 
cations of one another. They differed in experimental design and execution, setting, and the type of com- 
puter applications investigated. To confound matters, evaluation findings or results tended to differ from 
one investigation to another. Findings from different studies differed from each other, with some studies 
producing contradictory results. As well, many of the reviews are typically narrative and discursive in pre- 
sentation. resulting in their multiplicity of findings not capable of being absorbed by the reader without 
quantitative methods of reviewing (Kulik. Bangert, & Williams, 1983). 

Because of the shortcomings of the traditional approach of narrative reviews of research studies, attempts 
have been made to identify more promising methods of research investigation and research evaluation. 
Glass (1976) was the first to deal with the information overload problem by introducing a novel and com- 
prehensive method that allows one to estimate the average effect of treatments on outcome variables 
across numerous studies. He coined the term “meta-analysis,” and distinguished it from primary analysis 
and secondary analysis. 
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Primary analysis is the original research that includes data collection, data processing, and publication of 
results. Secondary analysis requires a different investigator who, following the same research question, 
conducts an analysis of the original data from either a different perspective or with different techniques. 
Meta-analysis draws upon the summary statistics of a variety of studies without having access to the origi- 
nal data. According to Glass, the aim of meta-analysis is to integrate a large number of results, with the 
focus not on statistical significance but on the size of treatment effects (Schwarzer, 1998). 

In 1977, Hartley was the first to apply meta-analysis to findings on CAI. Her study focused on mathemat- 
ics education in elementary and secondary schools. She reported that the average effect of CAI was to 
raise student achievement by .41 standard deviations, or from the 50th to the 66th percentile. She also 
reported that the effects of CAI were not so large as those produced by programs of peer and cross-age 
tutoring. However, they were far larger than effects produced by programmed instruction or use of indi- 
vidual learning packets. As well, Hartley discovered only small effects of study features on study out- 
comes (Kulik, Bangert, & Williams, 1983). 



PREL Meta-Analysis 

r I ^his meta-analysis conducted by Pacific Resources for Education and Learning (PREL) attempts to 
shed further light on the effectiveness of CAI on student achievement in reading by synthesizing 
diverse studies that have been conducted on the topic. The problem being addressed is: What is the effect 
of CAI on the reading achievement of students in grades K-12? 

This problem is posed within the larger context of whether or not computers present a workable method 
of instruction — that is, are they efficient and cost effective as an educational tool? Educators hope that 
computer use will inspire children turned off by traditional paper and pencil methods to achieve at levels 
beyond those currently being achieved. Concerns for fiscal feasibility may become negligible if it can be 
demonstrated that children exposed to computer-assisted instruction are happier, more productive mem- 
bers of society, gaining more academically and becoming better equipped to compete globally through 
computer use (Hamilton, 1995). With technology and software changing so rapidly, researchers must con- 
tinue to explore their different aspects on achievement. 



Major Challenges 

T wo major challenges were encountered in conducting this meta-analysis. First, the process is 
described through a plethora of terminology. This diversity includes computer-assisted instruction 
(CAI), computer-based instruction (CBI), computer-based learning (CBL), computer-based teaching 
(CBT), computer-managed instruction (CMI), and a generous sprinkling of other terms. For the purpose 
of this report, computer-assisted instruction (CAI) is used consistently. 

The second challenge refers to the rapidly evolving nature of computer technology. From reliance on the 
use of a mainframe computer in the Stanford project (Stanford Computer Assisted Instruction) to develop 
programs capable of individualizing reading instruction for students in kindergarten through third grade 
(Singhal, 1998) to use of individual computer stations for students independent of a main frame, the field 
is now characterized by a tremendous range of uses of computer technology that includes sophisticated 
WEB-based distance learning and hypermedia. Hypermedia programs allow the user to integrate sound, 
animation, graphics, and text through a variety of paths into one document. Hypermedia was designed to 
allow the student control of his own learning by using a variety of stimuli and his own interests as guides 
(Hamilton, 1995). Indeed, developments in computer technology have been occurring so swiftly that one 



would be hard pressed to predict with confidence what the next few years will bring in computer-assisted 
instruction (Kulik, Bangert. & Williams, 1983). 

In reviewing the integrative analysis that follows, the reader may easily lose touch with the kinds of 
research being integrated. The statistics and graphs that represent the findings of this meta-analysis of the 
impact of CAI on reading achievement will seem very remote from the studies themselves. And, in a real 
sense, the statistical manipulations carried out in order to arrive at general conclusions may undoubtedly 
place the reader in a position of losing qualitative or personal familiarity with the research (Glass. 
McGaw, & Smith, 1981). 



Objectives and Hypotheses 

r J 1 his meta-analysis seeks to answer the following questions: 



• How effective is computer-assisted instruction in teaching students to read? 

• Is it especially effective for certain types of outcomes or certain types of students? 

• Under what conditions is computer-assisted instruction most effective for the teaching of read- 
ing? 

In the research reported here, an attempt has been made to correct technical shortcomings of available 
research studies in order to determine if the huge body of research literature on reading achievement real- 
ly is hopelessly confusing, or whether the messages are merely buried in myriad results awaiting discov- 
ery through application of more advanced methods of research investigation. An underlying belief is that 
computer-assisted instruction and computer programs to teach reading may hold great promise for becom- 
ing powerful instructional tools that increase students’ engagement in reading, enhance reading compre- 
hension. and improve reading skills (Singhal, 1998). 



Methodology 

I n this section, the methods described are those by which the studies were identified, selected for this 
review, and coded; and the quantitative findings integrated. 



Literature Search 

T he literature search for this report was carried out in three phases: (1) document retrieval and 

abstracting resources; (2) previous reviews of the CAI and reading achievement literature; and (3) the 
bibliographies of studies once found, including footnote chasing. 

Studies were obtained by initially conducting computerized database searches of Educational Resources 
Information Center (ERIC) from September 1982 to September 1998, and PsycINFO from 1982 to 1998. 
Key terms entered for these databases were “reading and computer,” “reading and CAI,” and “reading 
achievement and CAI.” Studies that were already known to the researchers were also included. In addi- 
tion, footnote chasing was carried out as part of the reading process, and bibliographies of the study 
reports that were read were carefully reviewed to collect additional studies. A total of 33 studies were 
identified and collected for this analysis. 
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Criteria for Inclusion 



T° be included in this analysis, each study had to meet the following criteria: 

The study was published between January 1982 and January 1999 

• ne study repon conamed sufficient dat, for calculations in ,he meta-analysis, 
lne study focused on the effect of computer usa?e fPAT/ppn a * i 

v», b ,es w as telated to reading ** “ 

• 1 he subjects in the study were students in grades K- 12. 



Of the 33 studies collected, 17 studies 
sis. 



met the above criteria and served as the basis for this 



meta-analy- 



Coding of Studies 



I n case there was variation among the effect nf j* 

such variation. Therefore. H ’ “ ? trace *e cause of 

teristics for this analysis were as follows: U C ° ^ 7116 pnmary sludy charac ' 



Sampling method (some form of random sampling vs. convenient sampling) 

Control group (present or absent) 

Duration of treatment (one week, one month, one semester etc ) 

“ - “ inori,y y -ps. educationally disad- 

Software used for CAI (commercial software or not) 

Computer platform used (Macintosh, DOS, Windows, Mainframe etc.) 

Instniment used to measure reading achievement (standardized or not) 

Grade level of subjects 9 

Actual sample size 



Published year of study repon 

Statistics used (e.g., Chi-Square, t test, F test) 

Publication type (thesis/dissertation, journal, etc.) 



Three researchers at Pacific Resources for Education and Learning tPRFi \ H e 

study characteristics. Then, studies included in rhic mhtu , . 8 discussed how to code the 

researchers met again to compare the results of the corf ^ ^ mdependenl, y’ and the three 

ences in coding , 'chain ,he Z, c^g tsl ' ^ ““ <WS ' d and diff “- 



Analysis and Findings 



Description of Selected Studies 



ERIC 



S eventeen studies met the criteria for inclusion in this meta-analv.k Th, u • , 

are presented in Tables 1 - 7. Of the 17 studies since 198? 4 i ^ , aractenst,cs of the studies 

ThbK 3). Subjco, m of 






to 



or were educationally disadvantaged (see Table 4). The majority of the studies (53%) used sample sizes 
less than 50 (see Table 5). About 18 percent of the studies were conducted with students in high schools 
(see Table 6). In 65 percent of the studies, treatments were between five months to one school year, while 
24 percent were short-term, lasting four months or less (see Table 7). The 17 studies included in this meta- 
analvsis are listed at the end of this report. 



Table 1 

Study Reports by Year of Publication 



Publication Year 


Number of Studies 


Percentage of Studies 


1982-85 


j 


6 


1986-89 


4 


24 


1990-93 


5 


29 


1994-97 


7 


41 


Total 


i 17 


100 



Table 2 

Study Reports by Publication Source 



Source 


Number of Studies 


Percentage of Studies 


ERIC Document 


11 


65 


Journal 


5 


29 j 


Thesis 


l 


6 


Total 


17 


100 



Table 3 

Study Reports by Instrument Used 



Instrument 


Number of Studies 


Percentage of Studies 


Standardized 


15 


88 


N on-S tandard ized 


1 


6 


Unknown 


1 


6 


Total 


17 


100 



Table 4 

Study Reports by Population of Subjects 



Special Population 


Number of Studies 


Percentage of Studies 


Minority /M i grant 


4 


24 


Educationally Disadvantaged 


3 


18 


Low Socio-Economic Status 


4 


24 


Rural 


1 


6 


Urban/Suburban 


2 


12 


Other 


3 


18 


Total 


17 


102* 



* Note: Percentages may not add to 100 due to rounding. 
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Table 5 

Study Reports by Sample Size 



Sample Size 


Number of Studies 


Percentage of Studies 


Less than 30 


1 


6 


30-49 


8 


47 


50-99 


4 


24 


100-199 


2 


12 


200 or more 


2 


12 


Total 


r i7 


101 * 



* Note: Percentages may not add to 100 due to rounding. 



Table 6 

Study Reports by Grade Level of Subjects 



Grade Level 


Number of Studies 


Percentage of Studies 


1-2 


2 


12 


2-8 


2 


12 


3 


1 


6 


3-6 


2 


12 


3-8 


1 


6 


4 


2 


12 


5 


1 


6 


6 


2 


12 


7 


1 


6 


9 


2 


12 


10-11 


1 


6 


Total 

* r-\ . 


17 


102* 



* Note: Percentages may not add to 100 due to rounding. 



Table 7 

Study Reports by Treatment Duration 



Treatment Duration 


Number of Studies 


Percentage of Studies 


1 - 4 months 


4 


24 


5 months - 1 SY 


11 


65 


More than 1 SY 


2 


12 


Total 


17 


r lot * 



* Note. Percentages may not add to 100 due to rounding. 



o 

ERIC 



BESTCOPY AVAILABLE 



Computation of Effect Sizes 

M ost of the studies included tests of significance and significance levels. However, reports on tests of 
significance were more accurate than those on the levels of significance. For example* when a study 
reported t value* degree of freedom* and p value as 8.53, 45, and p < .01 respectively, it would be more 
accurate to estimate the effect size based on the t value and its degree of freedom than estimating from 
p = .01. Therefore* tests of significance were used to compute the effect sizes in this meta-analysis. 

In deciding whether the d-type effect size or r-type effect size should be used in this study, the researchers 
decided to use the r-type effect size, primarily because r-type effect size is more useful for the following 
reasons: 



• Given a d-type effect size* r makes perfectly good sense in its point biserial form if the indepen- 
dent variable has just two levels. 

• The r-type effect size requires no computational adjustment in going from the two-sample or 
multi-sample to the one sample case. This is not the case with the d-type effect size. 

• R-type effect size can be interpreted more simply in terms of practical importance than can d- 
type effect size (Cooper & Hedges, 1994). 



It is a well-known fact that the farther the r value of a population is away from zero, the more the distrib- 
ution of r values sampled from that population becomes skewed. This will complicate the combination of 
r values (Fisher, 1928). If r values are transformed into Fisher’s Zr values, the distribution will be nearly 
normal (Cooper & Hedges, 1994). Therefore, each r-type effect size was later transformed into Fisher’s Zr 
using the following formula: 



Z r 




1 + r 
1-r 



Most of the studies reported t test results for independent groups. In those cases, effect sizes were comput- 
ed using the following formula: 




In very rare cases, effect sizes were computed from the raw data when raw data were provided and the 
test used was deemed inappropriate in computing the desired effect size. 



Combining Effect Sizes 



T wo steps were taken prior to the computation of the overall effect size. First, it was decided that effect 
sizes with larger samples should be more heavily weighted. Thus, each Zr value was multiplied by 
(n - 3) to obtain the weighted effect size, where (n - 3) was the inverse of the conditional variance of Zr. 




Second, the problem of studies with more than one test of significance had to be resolved. In the current 
analysis, some studies employed more than one test of significance yielding more than one effect size. Thus, 
the 17 studies in this meta-analvsis yielded 40 effect sizes, as shown in Table 8. If all of these 40 effect sizes 
were used to compute the overall effect size of CAI on reading achievement, studies contributing more than 
one effect size would have more weight on the outcome of the meta-analysis. Rosenthal’s recommendation 
is to have each study contribute only a single effect size (Rosenthal, 1991, p. 27). Therefore, for each study 
yielding more than one effect size, an overall effect size for that study was computed. 
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Table 8 

Forty Individual Effect Sizes from 17 Studies 



No. 


Study 


Sample 

n 


Effect Size 

z, 


Weighted ES 

(n - 3)Z r 


1 


Marcinkiewicz. 1988 


30 


0.147 


3.971 


2 


Ngaiyaye & VanderPloge, 1986 


138 


0.191 


25.762 


3 


Ngaiyaye & VanderPloge, 1986 


190 


0.108 


20.180 


4 


Ngaiyaye & VanderPloge, 1986 


137 


0.027 


3.655 


5 


Ngaiyaye & VanderPloge, 1986 


116 


0.039 


4.449 


6 


Hamilton, 1995 


46 


0.054 


2.333 


7 


Jones, 1993 


30 


0.129 


3.496 


8 


Saracho, 1982 


256 


0.172 


43.434 


9 


Peak & Dewalt. 1993 


50 


0.243 


11.440 


10 


Mathis, 1996 


60 


0.244 


13.924 


11 


Hardman. 1 994 


42 


0.247 


9.644 


12 


Greenlee-Moore & Smith, 1994 


31 


0.254 


7.099 


13 


Davidson. Elcock & Noyes, 1996 


60 


0.313 


17.855 


14 


Williams, 1993 


108 


0.472 


49.611 


15 


Arroyo, 1992 


30 


0.621 


16.761 


16 


Tillman, 1995 


30 


0.762 


20,566 


17 


Casteel, 1989 


20 


0.228 


3.875 


18 


Heise, Papalewis. & Tanner, 1991 


56 


0.249 


13.198 


19 


Heise, Papalewis. & Tanner. 1991 


56 


0.030 


1.575 


20 


Reitsma, 1988 


35 


0.255 


8.149 


21 


Reitsma, 1988 


35 


0.121 


3.868 


22 


Reitsma, 1988 


35 


0.118 


3.777 


23 


Reitsma, 1988 


35 


0.311 


9.940 


24 


Reitsma. 1988 


34 


0.221 


6.855 


25 


Reitsma, 1988 


34 


0.096 


2.988 


26 


Reitsma. 1988 


35 


0.084 


2.694 


27 


Reitsma. 1988 


35 


0.062 


1.977 


28 


Reitsma, 1988 


34 


0.074 


2.298 


29 


Paul, Swanson, Zhang, & Hehenberger, 1997 


689 


0.089 


60.981 


30 


Paul, Swanson, Zhang, & Hehenberger, 1997 


689 


0.209 


143.111 


31 


Paul, Swanson, Zhang, & Hehenberger, 1997 


687 


0.079 


54.004 


32 


Paul, Swanson, Zhang, & Hehenberger, 1997 


687 


0.220 


150.283 


33 


Paul, Swanson, Zhang, & Hehenberger, 1997 


672 


0.023 


15.657 


34 


Paul, Swanson, Zhang, & Hehenberger, 1997 


672 


0.196 


130,928 


35 


Paul, Swanson, Zhang, & Hehenberger, 1997 


504 


0.065 


32.61 1 


36 


Paul, Swanson, Zhang, & Hehenberger, 1997 


504 


0.167 


83.430 


37 


Paul, Swanson, Zhang, & Hehenberger, 1997 


399 


0.004 


1.501 


38 


Paul, Swanson. Zhang, & Hehenberger, 1997 


399 


0,131 


52.051 


39 


Paul, Swanson, Zhang, & Hehenberger, 1997 


395 


0.077 


30.149 


40 


Paul, Swanson, Zhane, & Hehenberger, 1 997 


395 


0.155 


60.863 







Table 9 shows the resulting 17 calculated composite effect sizes from the 17 studies. 



Table 9 

Calculated Composite Effect Sizes of 17 Studies 



No. 


Study 


Sample 

n 


Effect Size 

Z 


Weighted ES 

(n - 3)4 


1 


Marcinkiewicz. 1988 


30 


0.147 


3.971 


2 


Ngaivaye & VanderPlose. 1986 


148 


0.095 


54.046 


3 


Heise. Papalewis. & Tanner. 1991 


56 


0.139 


14.773 


4 


Reitsma. 1988 


35 


0.045 


12.683 


5 


Paul. Swanson. Zhans. <5c Hehenbereer. 1997 


558 


0.123 


815.569 


6 


Hamilton. 1995 


46 


0.054 


2.333 


7 


Jones. 1993 


30 


0.129 


3.496 


8 


Saracho. 1982 


256 


0.172 


43.434 


9 


Peak & Dewalt. 1993 


50 


0.243 


11.44 


10 


Mathis. 1996 


60 


0.244 


13.924 


11 


Hardman. 1994 


42 


0.247 


9.644 


12 


Greenlee-Moore & Smith. 1994 


31 


0.254 


7.099 I 


13 


Davidson. Elcock. & Noyes. 1996 


60 


0.313 


17.855 


14 


Williams. 1993 


108 


0.472 


49.611 


15 


Arroyo, 1992 


30 


0.621 


16.761 


16 


Tillman. 1995 


30 


0.762 


20.566 


17 


Casteel. 1989 


20 


0.228 


3.875 



From Table 9, the overall effect size of 0.1316 was obtained by using the following formula: 

I(/?-3)Z r 

Standard error of the Overall Z * was computed using the following formula and was found to be 0.0109. 

Standard Error « 

I (n-3) 



Thus, the overall effect size of 0.1316 was significantly different from zero since Z for the studies combined 
= 0.1316/0.0109 = 12.04, which exceeded the critical value of 1.96 for a = .05 in the standard normal distri- 
bution. The lower and upper limits of the 95% confidence interval of the overall effect size were found to be 
Ll = 0.1316 - [1.96(0.0109)] = 0.1101 and Lu = 0.1316 + [1.96(0.0109)] = 0.1530 respectively. 



For interpretability, the overall effect size Zr and its lower and upper 95% confidence limits were trans- 
formed back into estimates of correlations using the formula r = (e 22 - ^/(e 22 + 1). It was found that the 
overall correlation and its lower and upper limits were 0.1308, 0.1097, and 0.1518 respectively. These 
results indicate that CAI promotes higher achievement in reading than instruction without CAI. 



Figure 1. 



Stem-and-leaf plot of the 
sample distribution of 
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However, a closer look at the stem-and-leaf plot of effect sizes in 17 studies (Figure 1) showed that they 
looked very heterogeneous. Thus, a test of homogeneity of the effect sizes was performed. Using the for- 
mula Q = S[(Zr - overall effect size) 2 (n - 3)] t the value of Q for homogeneity was found to be 182.236, 
which exceeds the critical value of the chi-square distribution for k = 17 - 1 = 16 degrees of freedom. 
Thus, the 17 effect sizes in the studies are clearly heterogeneous. 

To understand the reasons for the heterogeneity of effect sizes, the researchers consulted the studies to 
search for the study characteristics that might have caused the variation among the studies. Since the num- 
ber of included studies was only 17, scatter plots were used to check whether there was systematic varia- 
tion among the effect sizes due to some characteristics of the 17 studies. Figures 2 - 4 indicate that there 
seem to be no systematic variation among the effect sizes due to the sample size, the duration of CAI 
treatment, or the grade level of students in the 17 studies. 




Figure 2. Effect size (estimated Fisher’s Zr between CAI and Reading Achievement) displayed 
as function of sample size (n = 17) 




Figure 3. Effect size (estimated Fisher’s Zr between CAI and Reading Achievement) 
displayed as function of duration of treatment (n = 17) 



CORRELATION (Z.) 



0.8 




1 = Grades 1-8,2 = Grades 9-12 

Figure 4. Effect size (estimated Fisher’s Zr between CAl and Reading Achievement) 
displayed as function of grade level of subjects (n = 17) 

The File Drawer Problem 

T here is suspicion among statisticians and researchers that 5 percent of the pub- 
lished studies contain significant results that in reality are not significant, while the 
majority of the unpublished studies contain nonsignificant results (Rosenthal, 1979; 
Rosenthal & Rubin. 1988). Therefore, many believe that the published studies are a 
biased sample of the studies actually carried out (Bakan, 1967; McNemar, 1960; 

Smart, 1964; Sterling, 1959). In this meta-analysis only 17 studies were included, and 
the finding was that CAI promotes higher reading achievement. What would happen to 
this finding if there were studies that could have been included in this meta-analysis, 
but were not? 

There are three possible types of non-retrieved or new studies for this meta-analysis: 

1. The studies in which the average result is that CAI promotes reading achieve- 
ment (i.e., positive effect). These studies would not change the finding of this 
meta-analysis. Thus, there should be no concern about them. 

2. The studies in which the average result is that CAI is harmful for reading 
achievement (i.e., negative effect). There was no indication of such a finding 
among the 17 studies in this meta-analysis. Theoretically, there is no indica- 
tion that CAI is harmful for reading achievement. 

3. The studies in which the average result is that CAI does not promote reading 
achievement. These studies could change the finding of this meta-analysis. 
Therefore, this type of study always concerns meta-analysts. 

The number of non-retrieved or new studies indicating no CAI effects that would be 
required in order to change the conclusion that CAI promotes higher reading achieve- 
ment was estimated using the formula: 

X = [K(K(Mean Z) - 2.706] / 2.706 ] where X is the number of non-retrieved or new 
studies and K is the number of studies in the meta-analysis (Rosenthal, 1991, p.104). 

In the above formula, mean Z of the 17 studies was substituted with the expression (Z 
for the studies combinedWK where K is the number of studies in the meta-analysis. 



It was found that a total of 893 non-retrieved or new studies, averaging no CAI effect results, are neces- 
sary to conclude that there is no evidence of CAI effect on reading achievement. 



Conclusions 



Summary 

T his study tried to statistically combine the results of the studies that dealt with the important question of 
whether computer-assisted instruction is effective in raising the reading achievement of students in K- 
12. Literature searches were carried out using the ERIC database, the PsycINFO database, and relevant 
Internet sites followed by footnote chasing. Seventeen studies met the criteria for inclusion in the meta- 
analysis. Study characteristics were coded by three meta-analysts at Pacific Resources for Education and 
Learning. 

The method of meta-analysis was employed in the following order, (i) The r-type effect sizes were com- 
puted. (ii) These correlations were transformed into Fisher’s Zr values, (iii) The Zr values were weighted 
by the sample size of each study, (iv) The overall effect size Z and its lower and upper 95% confidence 
limits were computed and found to be 0.1316, 0.1101, and 0.1530 respectively, (v) The overall effect size 
Zr was tested using the standard normal distribution. It was found to be highly significant (Z = 12.04, p < 
.0000). (vi) For interpretability, the overall effect size Zr and its lower and upper limits at 95% confidence 
level were transformed back into estimates of correlations between CAI and reading achievement. The 
lower and upper limits were found to be 0.1097 and 0.1518 respectively. The fact that 95% confidence 
interval did not include zero confirmed the finding that the effect of CAI on reading achievement was sig- 
nificantly higher than zero, (vii) The tolerance of this finding was checked for non-retrieved or new stud- 
ies. Using Rosenthal’s formula, the number of studies not included in this meta-analysis that could change 
the current finding, provided that their findings indicated no CAI effect on reading achievement, was 
found to be 893. Thus, the finding of this meta-analysis was deemed highly tolerant. 

A test of homogeneity revealed that the effects of CAI on reading achievement in 17 studies were not 
homogeneous. Scatter plots were used in an attempt to find the cause of variation among the effect sizes 
in 17 studies. But, there seemed to be no particular study characteristic that might have caused the varia- 
tion systematically. 



Discussion 

T he overall finding of this meta-analysis is that computer-assisted instruction has a positive impact on 
reading achievement. However, there is a wide range in the foci, procedures, materials, and findings 
among the studies included in this meta-analysis. In some cases, a scarcity of acceptable studies was evi- 
dent in many categories. Therefore, the results given here must be interpreted with caution until a greater 
number of similar studies with similar reporting styles is available to confirm or refute the findings. 

Lack of sufficient numbers of studies in key areas could perhaps be the greatest barrier to the systematic 
assessment of the impact of CAI on the teaching of reading. Findings indicate that computer applications 
can play a significant role in teaching and learning. However, the precise nature of that role still needs to 
be researched with greater depth and precision. 

Learning is a complex endeavor. Therefore, it goes without saying that the use of CAI alone may be insuf- 
ficient in the teaching of reading. While CAI as an instructional tool has been effective in raising reading 



achievement, especially when used to supplement traditional instruction, other variables need to be con 
sidered in the teaching of reading. 



Implications 

I n light of the finding that computer-assisted instruction (CAI) promotes higher reading achievement, it 
is important to note the implications of this meta-analysis. Consideration could be given to programmat- 
ic implications as well as to implications for further research by posing the following questions: 

1. What currently available software focuses on the improvement of reading achievement for grades 
K-12? 

2. What developmental products are on the horizon that would support CAI instruction to increase 
reading achievement? 

3. What do schools need to have in place to provide equal access to CAI in reading for all students? 

4. What components of a reading curriculum and of reading instruction provide the most positive 
results with CAI? 

5. What effect does CAI have on reading achievement for special populations (i.e., 
minority/migrant, educationally disadvantaged, low socio-economic status)? 

6. What are the most effective strategies for implementing CAI in reading (i.e., stand-alone vs. inte- 
grated instruction; in-classroom vs. pull-out program; length of time for instruction; distance 
learning; web-based instruction)? 

7. What is the role of the teacher in the effective use of CAI? 

8. What implications do new technologies have for CAI in reading? 

Computer applications to teach reading hold great promise as instructional tools to increase students’ 
engagement in reading, promote reading comprehension, and improve reading skills. CAI can assist teach- 
ers in developing a more individualized approach to reading instruction to meet the diverse range of stu- 
dents’ needs in classrooms. Teachers can be empowered to vary the pace of instruction, review student 
learning, teach and reinforce specific skills and strategies, improve motivation, and provide students with 
relevant and timely feedback. 

Reading instruction aligned with computer-assisted instruction can serve as a powerful teaching tool to 
assist teachers in helping students reach their potential in reading. 
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